Recursion and Recursion- Like Structure in Ensembles of Neural Elements

Connectionist models have used general principles to model phenomena across many mental domains. Thus, they seem promising for uniting diverse syntactic phenomena (e.g., language, music, action). A challenge has been understanding how recursion can work in neurons. Headway has been made in training Recurrent Neural Networks (RNNs) to process languages which can be handled by the use of one or more symbol-counting stacks (e.g., ab, aABb, abc). Success with exponential state growth languages, in which stacks function as full-fledged sequence memories (e.g., palindrome languages, certain natural language relative clause constructions), has not been as great. [24] introduces Fractal Learning Neural Networks (FLNNS), showing that they can learn some exponential state growth languages with high accuracy. The current paper analyzes the performance of these FLNNs clarifying the relationship between their imperfect, but nevertheless structurally insightful, neural recursive encoding, and the perfect recursive encodings of symbolic devices.